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Abstract 


The  Joint  Advanced  Distributed  Simulation  Joint  Test  Force  (JADS  JTF)  is  chartered  by  the  U.S. 
Office  of  the  Secretary  of  Defense  (OSD)  to  determine  the  utility  of  advanced  distributed 
simulation  (ADS)  for  both  developmental  and  operational  test  and  evaluation  (DT&E  and 
OT&E).  The  program  is  at  its  midpoint,  and  this  paper  is  designed  to  provide  a  progress  report 
on  the  lessons  learned  to  date  on  the  use  of  ADS  in  test  and  evaluation  (T&E). 

The  paper  opens  with  a  brief  overview  of  ADS  technology  and  then  a  short  description  of  the 
JADS  Joint  Test  and  Evaluation  (JT&E)  program.  Third,  the  main  portion  of  the  paper  will 
discuss  the  results  and  lessons  learned  during  the  ADS-enhanced  testing  conducted  throughout 
the  first  major  phases  of  the  JADS  JT&E  program.  Fourth,  the  JADS  study  on  the  linking  of 
electronic  warfare  (EW)  test  facilities,  the  Threat  Systems  Linking  Architecture  (TSLA)  Study,  is 
briefly  described.  Finally,  other  considerations  will  be  offered  for  the  T&E  professional  interested 
in  whether  ADS  might  be  a  suitable  test  tool. 

The  material  in  this  paper  fuses  material  from  other  JADS  documents  prepared  by  many  members 
of  the  JADS  JTF.  Readers  are  encouraged  seek  more  information  JADS  has  compiled  either  via 
the  address  above  or  their  web  site  (http://www.jads.abq.com). 


Overview  of  ADS  (Ref.  1) 

Since  the  mid-1980s,  rapidly  evolving  information  systems  technology  has  been  put  to  work  in 
support  of  U.S.  Department  of  Defense 


(DoD)  needs.  Early  efforts  were  conducted  jointly  by  the  Defense  Advanced  Research  Projects 
Agency  and  the  U.S.  Army.  This  early  project  was  named  Simulation  Network  (SIMNET),  and  it 
was  sharply  focused  on  training  applications.  Conceptually,  the  project  was  directed  toward 
linking  training  devices  (simulators)  with  human  operators  in  the  loop  at  distributed  sites  in  a 
common  virtual  environment  in  near  real  time.  SIMNET  evolved  to  distributed  interactive 
simulation  (DIS),  a  technology  implementation  which  is  more  flexible  and  far  reaching.  Formal 
industry  standards  have  been  established  for  DIS.  In  turn,  DIS  is  giving  way  to  high  level 
architecture  (HLA),  a  technical  approach  championed  by  the  U.S.  Defense  Modeling  and 
Simulation  Office. 
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JADS  uses  a  more  generic  term  for  the  technology  -  ADS.  This  is  defined  as  the  technology  and 
procedures  that  provide  a  time  and  space  coherent,  interactive  synthetic  environment  through 
geographically  distributed  and  potentially  dissimilar  situations.  Any  combination  of  live,  virtual, 
or  constructive  simulation  of  people  and/or  equipment  can  be  used.  ADS  is  the  concept;  DIS  and 
HLA  are  applications  of  ADS. 


Overview  of  JADS  JT&E 


Background  (Ref.  1) 

Because  of  widespread  interest  in  using  ADS  technology  to  support  T&E,  the  JADS  JT&E 
program  was  nominated  for  feasibility  study  in  1993.  The  nomination  was  motivated  by  the  T&E 
community's  concern  about  long-standing  test  constraints  and  limitations,  and  the  potential  utility 
of  ADS  for  relieving  some  of  those  constraints  and  limitations.  However,  there  was  widespread 
skepticism  that  ADS  might  not  be  able  to  deliver  high-quality  data  demanded  by  the  T&E 
community.  The  Services  concurred  with  the  need  for  a  rigorous  examination  of  ADS  utility  to 
testing,  and  OSD’s  Director  of  Test,  System  Engineering  and  Evaluation  chartered  JADS  as  a  full 
joint  test  program. 

JADS  JT&E  Charter  (Ref.  2) 

The  basic  JADS  JT&E  program  was  chartered  in  October  1994  to  investigate  the  utility  of  ADS 
for  both  DT&E  and  OT&E.  More  specifically,  JADS  is  to  investigate  the  present  utility  of  ADS, 
to  identify  critical  constraints  in  using  the  technology,  to  develop  the  methodologies  in  using  ADS 
in  various  T&E  applications,  and  to  provide  growth  requirements  for  ADS  so  that  as  it  matures  it 
better  meets  the  needs  of  the  T&E  community. 

At  the  time  of  chartering,  OSD  tasked  JADS  to  investigate  the  possibility  of  specifically 
examining  ADS  utility  to  EW  T&E.  This  additional  facet  of  the  program  was  subsequently 
chartered  in  August  1996  (Ref.  3). 

Test  Approach 

To  accomplish  this  charter,  JADS  is  conducting  three  series  of  ADS-enhanced  tests  in  widely 
different  areas  to  determine  the  utility  of  ADS.  Representative  “systems  under  test”  are  used, 
ones  that  have  already  undergone  testing  and  have  been  fielded.  Significant  system  performance 
data  is  available  then  for  comparison  with  the  data  obtained  in  the  tests  introducing  ADS  as  a 
methodology.  The  three  specific  test  programs  are  the  System  Integration  Test  (SIT)  utilizing 
two  air-to-air  missiles  (AIM-9M  Sidewinder  and  AIM- 120  Advanced  Medium  Range  Air-to-Air 
Missile  (AMRAAM));  the  End-to-End  Test  (ETE)  using  the  Joint  Surveillance  Target  Attack 
Radar  System  (Joint  STARS)  as  a  representative  command,  control,  communications,  computer, 
intelligence,  surveillance  and  reconnaissance  (C4ISR)  system;  and  the  Electronic  Warfare  (EW) 
Test,  utilizing  the  ALQ-131  self-protection  jammer  (SPJ). 
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SIT  evaluated  the  utility  of  using  ADS  to  support  cost-effective  testing  of  an  integrated  missile 
weapon/launch  aircraft  system  in  an  operationally  realistic  scenario.  The  purpose  of  SIT  also 
included  the  evaluation  of  the  capability  of  the  JADS  Test  Control  and  Analysis  Center  (TCAC) 
to  control  a  distributed  test  of  this  type  and  to  remotely  monitor  and  analyze  test  results. 

SIT  consisted  of  two  phases,  each  of  which  culminated  in  fully  linked  missions.  The  missions 
simulated  a  single  shooter  aircraft  launching  an  air-to-air  missile  against  a  single  target  aircraft.  In 
the  Linked  Simulators  Phase  (LSP),  the  shooter,  target,  and  missile  were  all  represented  by 
hardware-in-the-loop  (HWIL)  laboratories.  LSP  testing  was  completed  in  November  1996.  In 
the  Live  Fly  Phase  (LFP),  the  shooter  and  target  were  represented  by  live  aircraft  and  the  missile 
by  a  HWIL  laboratory.  LFP  testing  was  completed  in  October  1997. 

Linked  Simulators  Phase.  The  LSP  test  concept  was  to  replicate  a  previous  AIM-9M-8/9  live 
fire  profile  in  an  ADS  configuration  and  compare  missile  results  for  the  LSP  trials  to  those  from 
the  live  fire  test.  The  LSP  test  configuration  is  shown  in  Figure  1 . 
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Figure  1.  Linked  Simulator  Phase  Configuration 


The  F/A-18  Weapon  System  Support  Facility  (WSSF)  at  China  Lake,  California,  and  the  F-14D 
Weapon  System  Integration  Center  (WSIC)  at  Point  Mugu,  California,  were  the  shooter  and 
target,  respectively.  The  shooter  “fired”  the  AIM-9  in  the  Simulation  Laboratory  (SIMLAB) 
HWIL  facility  at  the  target  which  could  respond  with  countermeasures.  Runs  were  controlled 
from  a  test  control  center  which  ensured  all  nodes  were  ready  for  each  run,  issued  start/stop 
directions,  and  processed  data  packets  for  real-time  analysis  of  system  performance.  Test  control 
was  exercised  from  the  Battle  Management  Interoperability  Center  (BMIC)  at  Point  Mugu  while 
the  JADS  Joint  Test  Force  was  physically  relocating.  Control  switched  to  the  JADS  TCAC  in 
Albuquerque,  New  Mexico  after  the  move  was  complete. 

Live  Fly  Phase.  The  LFP  test  concept  was  to  replicate  previous  AMRAAM  live  fire  profiles  in 
an  ADS  configuration  and  compare  missile  results  from  the  LFP  trials  to  those  from  the  live  fire 
tests.  In  the  LFP,  ADS  techniques  were  used  to  link  two  live  F-16  aircraft  (flying  on  the  Gulf 
Test  Range  at  Eglin  Air  Force  Base,  Florida)  representing  the  shooter  and  target  to  an  AMRAAM 
HWIL  laboratory  (also  at  Eglin)  representing  the  missile.  This  configuration  allowed  data  from 
live  sources  to  drive  the  HWIL  laboratory  for  more  realistic  missile  results  and  is  shown  in  Figure 
2. 
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Figure  2.  Live  Fly  Phase  Configuration 

Global  positioning  system  (GPS)  and  telemetry  (TM)  data  were  downlinked  from  the  aircraft  and 
passed  to  the  Central  Control  Facility  (CCF)  at  Eglin.  GPS,  inertial  navigation  system  (INS),  and 
tracking  radar  data  for  each  aircraft  were  combined  by  the  TSPI  (time-space-position  information) 
Data  Processor  (TDP)  in  the  CCF  to  produce  optimal  entity  state  solutions.  The  aircraft  entity 
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state  data  were  transformed  into  DIS  protocol  data  units  (PDUs)  and  transferred  to  the 
AMRAAM  HWIL  simulation  at  the  Missile  Simulation  Laboratory  (MISILAB)  over  a  T3  link. 

The  shooter  aircraft  “fired”  the  AMRAAM  in  the  MISILAB  at  the  target  and  provided  data  link 
updates  of  the  target  position  and  velocity  to  the  missile  during  its  flyout.  The  AMRAAM  seeker 
was  mounted  on  a  flight  table  and  responded  to  radio  frequency  (RF)  sources  in  the  MISILAB 
which  simulated  the  seeker  return  from  the  target,  the  relative  motions  of  the  target  and  the 
missile,  and  electronic  countermeasures  (ECM).  A  link  between  the  CCF  and  the  JADS  TCAC 
allowed  JADS  personnel  to  monitor  and  record  the  simulated  intercepts. 

End-to-End  Test 


The  ETE  uses  distributed  simulations  to  assemble  an  enhanced  environment  to  be  used  for  testing 
command,  control,  communications  and  computer  (C4I)  systems.  The  object  is  to  determine  if 
ADS  can  provide  a  complete,  robust  set  of  interfaces  from  sensor  to  weapon  system  including  the 
additional  intermediate  nodes  that  would  be  found  in  a  tactical  engagement.  The  test  traces  a 
thread  of  the  battlefield  process  from  target  detection  to  target  assignment  and  engagement  at 
corps  level  using  ADS.  Figure  3  illustrates  the  basic  test  architecture. 


Figure  3.  End-to-End  Test  Architecture 
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The  ETE  is  a  four-phased  test.  Phase  1  was  largely  developmental  -  constructing  the  various 
components  necessary  to  executing  later  phases  of  testing.  These  components  include  a  high 
fidelity  emulation  of  the  Joint  STARS  radar  processes,  called  Virtual  Surveillance  Target  Attack 
Radar  System  (VSTARS),  which  includes  both  moving  target  indicator  and  synthetic  aperture 
radar  modes  of  operation.  Phase  2  links  representative  entities  for  the  end-to-end  process  while 
the  “system  under  test”  is  in  a  laboratory  environment  enabling  JADS  to  explore  the  utility  of 
ADS  in  the  DT&E  and  early  OT&E  of  a  C4I  system.  Phase  3  hosts  VSTARS  on  board  the  actual 
Joint  STARS  aircraft  and  performs  final  integration  testing.  Phase  4  is  an  actual  live  open  air  test 
with  the  aircraft  airborne,  with  the  environment  augmented  by  ADS. 

Electronic  Warfare  Test  (Ref.  6) 

JADS  EW  Test  was  chartered  separately  by  OSD  to  examine  the  utility  of  ADS  in  EW  T&E.  To 
allow  JADS  to  conduct  a  broad  analysis  of  this  domain  and  remain  within  very  tight  fiscal 
constraints,  a  “multi-vectored”  approach  is  employed.  JADS  leveraged  off  the 

U.S.  DoD  High  Level  Architecture  (HLA)  Engineering  Prototype  Federation  for  lessons  learned 
in  constructing  and  implementing  a  distributed  architecture  for  EW  T&E.  At  the  bequest  of 
DoD’s  CROSSBOW  Committee,  JADS  directed  the  TSLA  Study,  which  delineates  how  to  link 
DoD’s  EW  test  facilities  using  the  HLA.  Third,  JADS  is  participating  with  the  U.S.  Army  in  its 
Advanced  Distributed  Electronic  Warfare  System  (ADEWS)  test,  a  concept  that  provides  EW 
effects  on  communications  gear  in  the  open  air  environment  without  the  actual  EW  open  air 
emissions.  Fourth,  JADS  offers  test  agencies  and  program  offices  “comparison  studies,”  where  a 
traditional  test  of  a  system  is  compared  with  an  ADS-enhanced  test  to  identify  potential  benefits  in 
test  thoroughness,  time  and  money.  JADS  Flag  Officer  Steering  Committee  directed  that  these 
studies  be  performed  after  the  JTF  has  performed  its  self-protection  jammer  (SPJ)  test. 

SP  J  Test  (Ref.  7) 

The  SPJ  test  has  been  designed  as  a  three-phased  test  focusing  on  the  U.S.  DoD  EW  test  process, 
and  utilizes  the  ALQ-131  as  its  “system  under  test.”  Phase  1  is  a  non- ADS  test  of  the  SPJ  on  an 
open  air  range  (OAR),  augmented  with  data  obtained  by  testing  the  ALQ-131  in  a  hardware-in- 
the-loop  facility.  The  purpose  of  this  test  is  to  establish  a  baseline  of  environment  and 
performance  data  which  will  be  used  to  develop  the  ADS  test  environment  for  the  following 
phases  and  will  be  the  basis  for  determining  the  validity  of  ADS  test  results.  Phase  2  is  a  test  of  a 
high-fidelity  real-time  digital  system  model  (DSM)  of  the  ALQ-131  linked  with  hardware-in-the- 
loop  terminal  threats  and  a  constructive  model  of  an  Integrated  Air  Defense  System  (IADS).  The 
threat  laydown  from  the  OAR  is  replicated  in  the  synthetic  ADS  environment  and  the  ALQ- 

131  will  be  flown,  via  a  scripted  flight  profile  developed  from  the  actual  OAR  flights,  through  the 
IADS,  engaging  the  high-fidelity  terminal  threats.  Phase  3  is  a  test  of  the  SPJ  installed  on  an 
actual  aircraft  located  in  an  Integrated  System  Test  Facility  (ISTF).  The  facility  will  be  linked 
with  hardware-in-the-loop  threats  and  the  constructive  model  of  the  IADS  using  the  same  threat 
laydown  as  the  previous  test  and  controlled  by  the  same  scripted  flight  profile.  Figure  4  illustrates 
Phases  2  and  3  of  the  SPJ  test. 
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Figure  4.  Self-Protection  Jammer  Phases  2  and  3 


JAPS  JT&E  Test  Results 


At  the  time  of  this  writing,  JADS  has  completed  one  phase  of  the  ETE  and  both  phases  of  SIT. 
As  the  first  phase  of  ETE  was  largely  developmental,  this  section  will  focus  on  SIT  results.  For  a 
schedule  of  JADS  test  execution,  refer  to  Figure  5). 


Phase  1 


Figure  5.  JADS  Test  Execution  Schedule 


7 


Linked  Simulators  Phase  Results  (Ref.  8) 


The  key  results  from  LSP  testing  were  as  follows: 

-  The  simulation  facilities  were  properly  linked,  and  the  missile  flyouts  were  valid  for  the 
target  representation  in  the  Simulation  Laboratory  (SIMLAB).  However,  this  target 
representation  differed  somewhat  from  the  target  data  originating  from  the  Weapon 
System  Integration  Center  (WSIC). 

-  The  manual  method  for  replicating  a  given  profile  resulted  in  very  good  run-to-run 
reproducibility  of  the  engagements. 

-  The  average  latency  of  all  entity  state  data  during  the  final  mission  were  relatively  small 
(<100  milliseconds  from  simulation  to  simulation)  and  consistent  run-to-run.  However, 
relatively  large  random  latency  variations  were  often  observed  which  resulted  in  an 
uncertainty  in  the  target  location,  as  perceived  in  the  SIMLAB. 

The  ADS  network  provided  ample  bandwidth  and  no  loss  of  connectivity  during  testing. 


There  were  no  significant  ADS-induced  errors. 

-  The  reliability  of  the  long-haul  network  was  very  good,  and  the  availability  of  the 
complete  LSP  ADS  configuration  was  on  the  order  of  85%. 

-  Test  control  procedures  were  refined  throughout  the  preparation  process  and  worked  well 
during  testing. 

Live  Fly  Phase  Results  (Ref.  9) 

The  key  results  from  LFP  testing  were  as  follows: 

-  The  live  aircraft  were  properly  linked  to  the  missile  HWIL  laboratory,  and  the  Missile 
Laboratory  (MISILAB)  generated  valid  AMRAAM  data  during  the  engagement. 

Accurate  time-space-position  information  (TSPI)  solutions  were  generated  by  the  TSPI 
Data  Processor  (TDP)  to  the  order  of  one  to  three  meters  in  position  and  one  meter  per 
second  in  velocity.  This  well  exceeded  MISILAB  accuracy  requirements. 

The  shooter  and  target  TSPI  data  were  properly  synchronized  to  each  other  and  to  the 
umbilical  and  data  link  messages  for  input  to  the  MISILAB  simulation. 

Latencies  during  testing  were  relatively  stable  and  consistent,  but  fairly  large.  The  total 
latency  of  the  MISILAB  simulation  was  about  3.1  seconds.  This  large  value  of  latency 
was  due  to  the  processing  and  buffering  of  the  TSPI  data  to  produce  accurate  and  smooth 
solutions  and  to  the  synchronization  technique  used. 

-  The  ADS  network  provided  ample  bandwidth  and  no  loss  of  connectivity  during  testing. 

-  There  were  no  significant  ADS-induced  errors. 

-  Test  control  procedures  worked  well  during  testing  with  centralized  test  control  exercised 
from  the  CCF. 


System  Integration  Test  Lessons  Learned 


LSP  Lessons  Learned 


LSP  lessons  learned  are  documented  in  much  detail  in  JADS  LSP  Final  Report  (Ref.  10)  and  are 
categorized  in  the  general  domains  of  technical  and  infrastructure  lessons  learned.  What  follows 
are  some  of  the  highlights. 

LSP  Technical  Lessons  Learned 


-  Accurate  coordinate  transformations  are  necessary.  They  must  be  verified  and  validated  at 
each  site  and  then  revalidated  during  end-to-end  testing  as  early  as  possible  in  the  test  phase. 

-  Quantitative  validation  has  limitations.  JADS  intent  was  to  quantitatively  verify  missile 
simulation  performance  against  live  fire  data.  However,  as  only  one  live  fly  event  was  available  to 
support  the  process,  a  modified  approach  including  both  quantitative  and  qualitative  methods  was 
used,  and  successfully  identified  invalid  results. 

-  Network  interface  units  (NIUs)  need  improvement.  NIUs  are  necessary  if  two  nodes  cannot 
communicate  directly  in  a  common  language.  They  can  be  a  major  source  of  both  errors  and 
processing  delays.  Better  direct  user  control  of  the  content  of  the  data  and  network 
communications  is  needed. 

-  Common  ADS-related  hardware  and  software  is  needed.  In  the  LSP,  it  was  difficult  to  get 
the  ADS  network  to  behave  in  a  uniform  fashion  due  to  the  many  different  types  of  interface 
hardware,  communications  equipment  (routers),  and  interface  software  versions. 

-  Latency  variations  were  significant.  Processing  delays  were  the  primary  culprit  here. 

-  Time  sources  must  be  synchronized  off  the  same  time  source  and  then  must  be  validated  at 
each  test  site  prior  to  project  operations  to  ensure  accurate,  synchronized  time  is  precisely 
recorded  at  each  test  site. 

-  Special  test  equipment  is  needed  for  check-out  and  verification  of  the  ADS  architecture. 
Without  this  equipment,  trial  and  error  becomes  the  norm  when  (not  if)  problems  crop  up. 

LSP  Infrastructure  Lessons  Learned 


-  The  requirements  for  an  ADS  test  must  be  clearly  defined  early  in  the  test  planning  phase. 
This  includes  user  requirements,  support  agency’s  stated  actions,  and  operations  security 
requirements.  Planning  and  coordination  details  will  be  much  more  involved  than  in  a  traditional, 
non- ADS  test. 

-  Get  “system  under  test”  experts  involved  from  the  beginning. 

-  Test  communications  requirements  must  be  addressed  early  in  the  test  planning  phase.  This  is 
necessary  to  ensure  effective  communications  during  the  test.  Also,  a  linked  test  should  have 
multiple  (more  than  two)  communications  nets  with  easy,  selectable  access  to  all  the  nets  from 
multiple  locations  within  the  site.  Finally,  the  capability  for  secure  video  teleconferencing  pays  big 
dividends  during  planning,  coordination,  and  post-test  debriefs. 

-  A  stepped  build-up  approach  should  be  used.  First,  a  systematic  check-out  of  the  stand-alone 
simulators  (live,  virtual  or  constructive)  is  needed.  Next,  direct  (non-DIS)  links  should  be  used 
during  test  build-up.  Finally,  structured  testing  of  the  network  must  be  performed  prior  to,  and 
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independent  of,  the  linked  testing  times  and  the  simulation  laboratories  to  validate 
transmission/reception  rates,  bandwidth  utilization,  latency,  data  transmission  and  reception,  etc., 
prior  to  commencing  project  test  periods. 

-  Linking  of  facilities  using  ADS  can  require  significant  facility  interface  hardware  and  software 
development.  ADS  implementation  is  not  “plug  and  play,”  at  least  for  some  time. 

-  Local  (on-site)  test  monitoring/control  should  be  used  prior  to  remote  test 
monitoring/control. 

-  Tight  control  of  the  aircrew  is  not  desirable.  Give  them  the  critical  parameters  and 
switchology  to  meet  the  test  objectives  and  allow  them  to  make  tactical  decisions,  fly  the 
“aircraft,”  operate  the  weapon  system,  etc. 

-  Additional  time  is  needed  before  the  beginning  and  after  the  end  of  each  testing  period.  One 
hour  is  recommended  for  set-up,  and  two  hours  at  the  end  for  data  logging,  data  archiving,  data 
transfer,  and  laboratory  reclassification. 

-  Briefings  are  needed  before  and  after  each  mission. 

-  Effective  data  management  is  needed,  as  ADS  can  generate  mountains  of  data.  A 
comprehensive  plan  will  clearly  identify  the  data  to  be  collected  at  each  site,  on-site  processing  of 
the  data,  and  data  to  be  transferred  to  the  analysis  center. 

-  Adequate  time  must  be  allowed  for  data  analysis  between  test  events.  Analysis  procedures 
should  be  rehearsed  to  better  understand  the  amount  of  time  needed  for  this  analysis. 

-  Configuration  control  is  essential.  This  one  obvious  area  was  one  of  great  challenge 
considering  the  many  sites  involved  and  the  multiple  uses  of  each  site. 

LFP  Lessons  Learned 


LFP  lessons  learned  are  documented  in  much  detail  in  JADS  LSP  Final  Report  (Ref.  11)  and  are 
also  categorized  in  the  general  domains  of  technical  and  infrastructure  lessons  learned.  Some  of 
the  highlights  are  listed  below. 

LFP  Technical  Lessons  Learned 


-  As  in  the  LSP,  a  major  lesson  learned  is  that  stand-alone  simulation  facilities  (for  live,  virtual 
or  constructive  entities)  can  require  significant  modifications  before  effective  linking  is  possible. 

-  Additionally,  linking  may  require  special  purpose  interfaces  so  as  to  accept  inputs  in  real  time. 
Development  of  such  units  must  be  factored  into  test  planning. 

-  Key  interfaces  need  realistic  integration  testing.  Replaying  data  from  a  recorded  mission 
worked  well  in  most  cases  (and  was  most  cost  effective);  however,  some  integration  testing 
required  a  live  mission. 

-  Early  definition  of  network  requirements  was  very  advantageous.  This  was  a  major  lesson 
from  LSP  that  JADS  took  advantage  of. 

LFP  Instrumentation  Lessons  Learned 


-  Changes  and  upgrades  to  aircraft  instrumentation  delayed  development.  Specially 
instrumented  aircraft  were  required  to  support  the  LFP  flights.  Due  to  the  small  number  of  such 
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aircraft,  the  LFP  schedule  was  very  sensitive  to  periodic  aircraft  phase  inspections,  software 
upgrades,  and  higher  priority  missions. 

-  Merging  several  TSPI  sources  was  advantageous.  Real-time  aircraft  inertial  navigation 
system  (INS)  and  global  positioning  satellite  (GPS)  data  were  combined  to  calculate  more 
accurate  kinematic  estimates.  When  combined  with  the  ground  radars,  solutions  of  one  to  three 
meters  in  position  and  one  meter  per  second  in  velocity  were  achieved. 

-  A  strong  program  manager  or  system  integrator  is  needed  to  oversee  facility  development, 
due  to  the  difficulty  in  coordinating  several  diverse  facilities  to  successfully  integrate  an  ADS- 
linked  configuration. 

-  Use  risk  reduction  tests  for  integration.  A  building  block  approach  was  used  successfully  to 
check  out  interfaces  at  the  lowest  level,  then  one  or  two  resources  at  a  time  were  added  to 
integrate  the  linked  configuration.  These  risk  reduction  tests  were  also  useful  for  developing 
analytical  tools. 

-  Several  subnetworks  should  be  used  for  voice  communications.  Three  voice  communications 
networks  were  needed  to  support  more  than  30  people  at  various  locations,  and  a  fourth  network 
could  have  further  aided  decision  making. 

-  Two-dimensional  displays  were  needed  at  each  node;  they  greatly  enhanced  the  situational 
awareness  of  the  participants. 

-  Existing  range  procedures  had  to  be  modified  for  ADS.  The  existing  test  procedures  were 
only  written  for  individual  facilities,  so  a  new  combined  checklist  was  created  for  ADS 
applications. 

-  Laboratory  replays  served  as  an  excellent  method  of  test  rehearsal. 

Other  Topics  for  Consideration 
Threat  Simulator  Linking  Activities  (TSLAt  Study  (Ref.  12) 

TSLA  is  a  study  chartered  by  the  U.S.  DoD’s  CROSSBOW  Committee  and  directed  by  the  JADS 
JTF.  The  TSLA  study  provides  an  ADS  Capabilities  Assessment  Report  which  describes  the 
utility  of  ADS  in  the  context  of  the  evolutionary  acquisition  process  interwoven  with  T&E.  At 
each  phase  of  the  acquisition  process,  the  conventional  and  ADS  methodologies  are  applied.  For 
each  phase,  the  necessary  test  facilities  are  delineated  and  the  differences  in  test  capabilities  noted. 
Test  facility  requirements  are  addressed  for  SPJs,  stand-off  jammers,  and  integrated  avionics.  As 
the  test  facility  requirements  are  reviewed,  any  improvements  needed  to  meet  the  requirements  of 
the  electronic  combat  test  process  are  noted.  Some  of  these  improvements  are  needed  without 
regard  for  ADS.  In  other  cases,  the  improvements  are  needed  only  to  support  ADS. 
Comparisons  of  capabilities,  with  and  without  ADS,  are  discussed.  General  assessments  of  the 
cost  impact  of  ADS  are  also  discussed. 

Requirements  for  and  the  impact  of  latency  are  also  discussed.  Latency  will  be  present. 
Depending  on  the  network  topology,  physical  communication  infrastructure,  and  network 
management  methods,  it  may  be  possible  to  achieve  a  tolerable  latency  for  most  applications. 
Latency  remains  the  greatest  technological  risk  to  the  successful  use  of  ADS  in  EW  T&E. 
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Considerations  for  the  T&E  Professional 


Although  JADS  still  has  another  year  of  ADS  testing  ahead  of  it,  there  are  several  overall 
considerations  which  have  become  evident.  What  follows  are  some  highlights,  as  recently  briefed 
to  JADS  Flag  Officer  Steering  Committee  (Ref.  13),  and  adopted  into  a  pamphlet  entitled 
“Emerging  Findings  from  JADS.” 

-  ADS  allows  one  to  link  live,  virtual,  and  constructive  players  based  on  need.  ADS  does  not 
mean  linking  together  several  constructive  simulations  to  make  a  bigger,  more  complex  model. 
Rather,  it  means  blending  live,  virtual  and  constructive  players  to  give  the  user  the  right  mix  of 
fidelity  and  realism  to  meet  specific  needs. 

-  Distribution  is  not  a  function  of  distance.  Latency  is  a  function  of  processing  and 
transmission,  and  processing  latency  dominates.  Transmission  latencies  are  predictable  and 
relatively  well  behaved.  Processing  latencies  can  be  problematic,  though,  and  require  a  thorough 
understanding  of  the  individual  sites  and  the  ADS  architecture.  This  holds  true  whether  the 
network  covers  a  continent  or  multiple  nodes  at  a  single  location. 

-  Validating  against  live  data  is  problematic.  Problems  include  the  quality  of  the  live  data  and 
the  lack  of  data  availability. 

-  Data  collection  is  different  from  traditional  T&E  and  training.  Generally  speaking,  an  ADS 
environment  is  easier  to  instrument  than  the  traditional  live  test  environment  and  provides  more 
trials  per  unit  of  time.  The  end  result  is  that  analysts  can  get  inundated  with  data.  In  addition, 
ADS  testing  requires  additional  data  to  be  collected  on  the  performance  of  the  networks  linking 
your  sites. 

-  ADS  cost  benefits  are  best  realized  over  the  entire  system  life  cycle.  JADS  has  performed 
some  cost  benefit  analyses  comparing  traditional  test  approaches  to  ones  utilizing  ADS.  In  many 
cases,  it  appears  as  if  ADS  could  save  time  and  money  (as  well  as  allowing  a  more  rigorous  test) 
in  just  the  test  phase  of  a  development  program.  However,  the  full  benefits  of  using  ADS  would 
be  realized  over  all  the  phases  of  the  acquisition  cycle,  from  requirements  development  to  training 
and  sustainment.  This  supports  the  concepts  advocated  in  both  simulation  based  acquisition 
(SBA)  and  Simulation,  Testing  and  Evaluation  Program  (STEP). 

-  ADS  allows  one  to  test  “differently.”  Adding  ADS  to  a  traditional  test  approach  provides 
only  a  fraction  of  the  value  ADS  can  bring  to  bear.  To  realize  the  full  capabilities  of  this  enabling 
technology,  one  will  construct  a  test  event  fundamentally  differently  than  its  traditional  forefather. 

-  To  a  certain  extent,  latency  is  manageable.  The  ADS  architectural  design  is  the  most 
determining  factor  of  latency.  The  tester  must  approach  network  design  from  a  requirements 
viewpoint.  Based  on  what  the  tester  is  trying  to  accomplish,  an  architecture  can  usually  be 
designed  which  balances  the  types  of  participating  assets,  fidelity  requirements,  and  tolerable 
latency. 

-  The  effect  of  latency  is  dependent  on  the  players  involved.  Latency  is  a  factor  when  the  tester 
is  trying  to  generate  closed-loop  interactions.  Again,  the  tester  must  approach  the  test  from  a 
requirements  viewpoint  in  determining  whether  an  ADS  architecture  can  provide  the  interactions 
needed  for  the  test  event. 
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Conclusions 


JADS  has  been  chartered  to  determine  the  truth  as  to  where  ADS  is,  or  is  not,  a  feasible  tool  for 
T&E.  JADS  JT&E  testing  is  well  underway,  and  the  early  evidence  is  that  ADS  can  bring  many 
benefits  to  the  table.  However,  one  must  be  fully  aware  of  the  inherent  limitations  of  the 
technology.  Also,  one  would  be  well  advised  to  learn  from  those  who  have  practical  experience 
in  using  ADS  in  the  T&E  arena.  Distributed  simulation  is  certainly  not  a  panacea  that  will  solve 
all  of  the  problems  and  meet  all  of  the  requirements.  However,  it  does  appear  to  be  a  powerful 
tool,  if  used  appropriately  and  intelligently,  and  should  be  considered  in  balance  with  other 
methodologies  when  developing  a  T&E  program. 
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Acronyms 


Following  are  acronyms  found  in  this  paper’s  figures  and  not  defined  in  the  body  of  the  paper. 

Figure  1 

A/C  -  Aircraft 
IR  -  Infrared 
MSL  -  Missile 

SMS  -  Stores  Management  System 
TGT  -  Target 

Figure  2 

AASI  -  Aircraft  Avionics  Simulation  Interface 
ECM  -  Electronic  Countermeasures 
RDL  -  Rear  Data  Link 
UMB  -  Umbilical 
TM  -  Telemetry 

Figure  3 

ACE  -  Analysis  and  Control  Element 
ASAS  -  All-Source  Analysis  System 
ATACMS  -  Army  Tactical  Missile  System 
DOCC  -  Deep  Operation  Coordination  Center 
FDC  -  Fire  Direction  System 
FSE  -  Fire  Support  Element 
GSM  -  Ground  Station  Module 
LOS  -  Line  of  Sight 
OK  -  Oklahoma 

SATCOM  -  Satellite  Communications 
SCDL  -  Surveillance  Control  Data  Link 

TRAC  -  U.S.  Army  Training  and  Doctrine  Command  Analysis  Center 
WSMR  -  White  Sands  Missile  Range 

Figure  4 

ECM  -  Electronic  Countermeasures 
IADS  -  Integrated  Air  Defense  System 
RF  -  Radio  Frequency 
TECH  -  Technique 

Figure  5 

FY  -  Fiscal  Year 
Qtr  -  Quarter 
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